Skip to content

fix(flows): paper_flow draft-schema + code-assembled UUIDs#88

Open
bluezdot wants to merge 8 commits into
LLMQuant:masterfrom
bluezdot:fix/paper-flow-draft-assembly
Open

fix(flows): paper_flow draft-schema + code-assembled UUIDs#88
bluezdot wants to merge 8 commits into
LLMQuant:masterfrom
bluezdot:fix/paper-flow-draft-assembly

Conversation

@bluezdot

@bluezdot bluezdot commented Jul 7, 2026

Copy link
Copy Markdown

Problem

paper_flow could not produce a Paper end-to-end against a live model — a defect the test suite never caught because it mocks the SDK (scripts/verify.sh stayed green). Running it for real surfaced three failure modes:

  1. Strict JSON schema rejected Paper. Agent(output_type=Paper) fails at Runner.run with UserError: Strict JSON schema is enabled… — key-independent. PaperTreeKnowledge.nodes: dict[UUID, TreeNode] maps to an open-ended additionalProperties object, which strict mode forbids.
  2. The model cannot reliably emit UUIDs. With a non-strict wrap, it returned semantic ids ("paper_001", "introduction") for the many UUID-typed fields (id, root_node_id, node_id, the nodes keys, Citation.tree_id/node_id).
  3. Even when instructed to emit UUIDs, it corrupts them. Prompt-patching only reduced the errors: the model hallucinated an invalid hex char (…-bdle-… for …-bd1e-…) and reused one UUID for two nodes (a duplicate JSON key that silently drops a node).

Asking an LLM to hand-generate dozens of unique, valid, cross-referenced UUIDs is structurally unreliable.

Fix

Stop the LLM from ever touching an identifier:

  • New leaf module quantmind/flows/_paper_draft.py defines an id-less DraftPaper (it is the root node: carries title/summary/content/citations/children directly) with nested DraftNode/DraftCitation.
  • assemble_paper(...) is a pure function that walks the draft, mints every uuid4(), wires parent_id/children_ids/root_node_id and each Citation.node_id/tree_id, and builds the real Paper.
  • paper_flow sets the Agent's output_type to AgentOutputSchema(DraftPaper, strict_json_schema=False) and derives provenance (SourceRef/as_of/ExtractionRef(flow="paper_flow", model=cfg.model)) in code rather than trusting the LLM.

Design rationale: docs/design/en/2026-07-07-paper-flow-draft-assembly.md.

Verification

  • scripts/verify.sh green: ruff, basedpyright, import-linter (apex contract intact), pytest — 247 tests, 89.79% coverage (floor 75%).
  • Live end-to-end on a real PDF now returns a valid multi-node Paper with no extra_instructions, code-owned provenance, and no schema/UUID errors.
  • A test pins the Agent's output_type to the non-strict DraftPaper schema so this regression cannot silently reappear behind mocks.

Notes

  • DoiIdentifier still raises NotImplementedError (OA-PDF resolver is a separate follow-up).
  • SourceRef.fetched_at/content_hash intentionally left None in this MVP; documented as follow-ups.

🤖 Generated with Claude Code

bluezdot and others added 8 commits July 7, 2026 14:52
Root-cause and fix design for paper_flow being unable to produce a Paper
end-to-end (strict-schema rejection + LLM UUID hallucination/reuse).
Chosen approach: nested id-less draft schema; code assembles all UUIDs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Transitional: unblocks Runner.run past strict-JSON-schema rejection of
Paper's dict[UUID, TreeNode]. Superseded by the draft-schema design.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fixes end-to-end extraction: the Agent no longer emits UUIDs (eliminating
strict-schema rejection and UUID hallucination/reuse); provenance is now
code-owned.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… validates

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ale doc

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant